How can we classify an image using AI? Answer is in this video :)
submitted by /u/neuronovesite_cz
[link] [comments]
-
Open
submitted by /u/neuronovesite_cz
[link] [comments]
Pianist AI> Artificial intelligence composer> Level 7
About a year ago, I started the pianist AI project with the aim of having an AI model that can generate piano pieces. Although the optimization is still in process, today, finally it seems the model has learned the basic concepts.
I have named the first piece of Level 7: Peace> https://youtu.be/rLW3KwCG41M
With the hope of better tomorrow….
submitted by /u/amin_mlm
[link] [comments]
( 1
min )
-
Open
How to choose the correct flow control scheme?
submitted by /u/Just0by
[link] [comments]
( 1
min )
KAIST Researchers Propose DIGAN: An Implicit Neural Representation (INR)-based Generative Adversarial Network (GAN) for Video Generation Using Machine Learning
Deep generative models have produced realistic samples in a variety of domains, including image and audio. Video generation has recently emerged as the next issue for deep generative models, prompting a long line of research to learn video distribution.
Despite their efforts, there is still a big gap between large-scale real-world recordings and simulations. The intricacy of video signals, which are continuously coupled across spatiotemporal directions, contributes to the difficulty of video creation. Specifically, most previous works have modeled the video as a 3D grid of RGB values, i.e., a succession of 2D images, using discrete decoders such as convolutional or autoregressive networks. However, because of the cubic complexity, such discrete modeling limits the scalability of created movies and misses the intrinsic continuous temporal dynamics.
Continue Reading My Article Summary On This Research
Paper: https://openreview.net/pdf?id=Czsdv-S4-w9
Github: https://github.com/sihyun-yu/digan
Project: https://sihyun-yu.github.io/digan/
https://reddit.com/link/t7f7ti/video/nicll8ujvll81/player
submitted by /u/No_Coffee_4638
[link] [comments]
( 1
min )
The Top 5 AI Articles of February on Hackernoon!
submitted by /u/OnlyProggingForFun
[link] [comments]
Check Out These Paintings Entirely Generated by AI
submitted by /u/MarS_0ne
[link] [comments]
How far are we from AGI?
submitted by /u/NoAstronomer5087 [link] [comments] ( 1 min ) -
Open
[R]: Federated Learning with Formal Differential Privacy Guarantees
submitted by /u/giugiacaglia
[link] [comments]
[D] What metrics determine if a PhD program/research lab is good?
Hi all, I'm looking for advice from those of you who have completed or are currently enrolled in a PhD program. What, in your opinion, makes a research lab/program stand out as being a desirable place to complete a PhD? Thanks submitted by /u/c9m9 [link] [comments] ( 2 min )[D] How should I find the tolerance for these processes using ML?
I am studying a chemical process. The process has 4 steps. Each step will have some error, the error could finalize contribute to failing the produce (final result). Now I need to find out what would be the acceptance range of each step. How should I approach this problem? I am thinking to use classification tree. Any better option? submitted by /u/superhero_io [link] [comments] ( 1 min )[D] The 2030 Self-Driving Car bet between Jeff Atwood and John Carmack
submitted by /u/hardmaru
[link] [comments]
[R] SeamlessGAN: Self-Supervised Synthesis of Tileable Texture Maps
submitted by /u/crp1994
[link] [comments]
( 2
min )
[D] Worthwhile books on ethical machine learning for someone with background knowledge
Recently I read Cathy O’Neils Weapons of Math Destruction book. Whilst I found the topic enjoyable and for the most part an engaging read, I was left with the feeling that it only skims the surface from a technical viewpoint and works as a good introductory piece on the dangers of machine learning. I’ve been working with machine learning techniques for a number of years now but I’m interested in delving into the ethical and XAI components of machine learning. Would you have any must-reads for this area, whether that be technical works or reflections on current ML usage? I’m also open to any AI books in general that you found particularly interesting. submitted by /u/BlueGrassGrapes [link] [comments] ( 1 min )[D] How do you stay abreast of the latest developments in the field of AI/ML/Big Data ?
As someone with ML Engineer/Data Engineer experience, my friends who are currently pursuing PhD in AI/ML/Big Data are my only source of updates on recent developments in AI/ML fields. Other than that, I've started to lurk around relevant subs on reddit and certain hashtags on LinkedIn to see what's going on in the industry as such. But I wonder how do you all stay informed? Are there any particular websites that you follow? How does the research community do it? submitted by /u/money_noob_007 [link] [comments] ( 3 min )[D] What language do you perform your machine learning algorithms in?
Excluding preprocessing and data visualisation View Poll submitted by /u/fouried96 [link] [comments] -
Open
Does any way to map the RAM coordinates to pixel coordinates for the Montezuma game in the gym?
I am working with the ram version of this game for a master thesis, and I need to know the location for certain items as the keys and the stairs. The coordinates of x and y of the RAM oscillate in the interval [0, 255]; however, the image's shape that I can visualize is (210,160), and it seems impossible to discover a way to map the ram coordinates to the pixel coordinates. I would be very grateful if anyone could help me with this topic. submitted by /u/SrPinko [link] [comments] ( 1 min )Determining Robot distance to goal: Mapless Navigation
Hi, I would like to how can we determine robot distance from the goal incase of mapless navigation when the model is implemented on real robot. In simulation we can used the coordinated to do so, but what about the actual implementation? Do we need some additional sensors? I am mainly interested in indoor navigation using only a camera and DRL. Ref paper: https://www.cs.ox.ac.uk/files/11898/SnapNav.pdf Look at the reward function Thanks submitted by /u/ahsol360 [link] [comments] ( 1 min )How to take gradient of log policy when actions are negative?
I am currently trying to train Bipedal Walker of OpenAI gym by using policy gradient approach. My action space contains 4 continuous actions, all ranging [-1.0, 1.0]. In this case, how can we calculate gradient of log policy when ``````` log(negative_action) gives nan value. submitted by /u/Better-Ad8608 [link] [comments] ( 1 min ) -
Open
KamNet: An Integrated Spatiotemporal Deep Neural Network for Rare Event Search in KamLAND-Zen. (arXiv:2203.01870v1 [physics.ins-det])
Rare event searches allow us to search for new physics at energy scales inaccessible with other means by leveraging specialized large-mass detectors. Machine learning provides a new tool to maximize the information provided by these detectors. The information is sparse, which forces these algorithms to start from the lowest level data and exploit all symmetries in the detector to produce results. In this work we present KamNet which harnesses breakthroughs in geometric deep learning and spatiotemporal data analysis to maximize the physics reach of KamLAND-Zen, a kiloton scale spherical liquid scintillator detector searching for neutrinoless double beta decay ($0\nu\beta\beta$). Using a simplified background model for KamLAND we show that KamNet outperforms a conventional CNN on benchmarking MC simulations with an increasing level of robustness. Using simulated data, we then demonstrate KamNet's ability to increase KamLAND-Zen's sensitivity to $0\nu\beta\beta$ and $0\nu\beta\beta$ to excited states. A key component of this work is the addition of an attention mechanism to elucidate the underlying physics KamNet is using for the background rejection.Sample Efficient Deep Reinforcement Learning via Uncertainty Estimation. (arXiv:2201.01666v2 [cs.LG] UPDATED)
In model-free deep reinforcement learning (RL) algorithms, using noisy value estimates to supervise policy evaluation and optimization is detrimental to the sample efficiency. As this noise is heteroscedastic, its effects can be mitigated using uncertainty-based weights in the optimization process. Previous methods rely on sampled ensembles, which do not capture all aspects of uncertainty. We provide a systematic analysis of the sources of uncertainty in the noisy supervision that occurs in RL, and introduce inverse-variance RL, a Bayesian framework which combines probabilistic ensembles and Batch Inverse Variance weighting. We propose a method whereby two complementary uncertainty estimation methods account for both the Q-value and the environment stochasticity to better mitigate the negative impacts of noisy supervision. Our results show significant improvement in terms of sample efficiency on discrete and continuous control tasks.Local Citation Recommendation with Hierarchical-Attention Text Encoder and SciBERT-based Reranking. (arXiv:2112.01206v2 [cs.IR] UPDATED)
The goal of local citation recommendation is to recommend a missing reference from the local citation context and optionally also from the global context. To balance the tradeoff between speed and accuracy of citation recommendation in the context of a large-scale paper database, a viable approach is to first prefetch a limited number of relevant documents using efficient ranking methods and then to perform a fine-grained reranking using more sophisticated models. In that vein, BM25 has been found to be a tough-to-beat approach to prefetching, which is why recent work has focused mainly on the reranking step. Even so, we explore prefetching with nearest neighbor search among text embeddings constructed by a hierarchical attention network. When coupled with a SciBERT reranker fine-tuned on local citation recommendation tasks, our hierarchical Attention encoder (HAtten) achieves high prefetch recall for a given number of candidates to be reranked. Consequently, our reranker requires fewer prefetch candidates to rerank, yet still achieves state-of-the-art performance on various local citation recommendation datasets such as ACL-200, FullTextPeerRead, RefSeer, and arXiv.Deep Learning in Human Activity Recognition with Wearable Sensors: A Review on Advances. (arXiv:2111.00418v5 [cs.HC] UPDATED)
Mobile and wearable devices have enabled numerous applications, including activity tracking, wellness monitoring, and human--computer interaction, that measure and improve our daily lives. Many of these applications are made possible by leveraging the rich collection of low-power sensors found in many mobile and wearable devices to perform human activity recognition (HAR). Recently, deep learning has greatly pushed the boundaries of HAR on mobile and wearable devices. This paper systematically categorizes and summarizes existing work that introduces deep learning methods for wearables-based HAR and provides a comprehensive analysis of the current advancements, developing trends, and major challenges. We also present cutting-edge frontiers and future directions for deep learning-based HAR.Extracting Clinician's Goals by What-if Interpretable Modeling. (arXiv:2110.15165v2 [cs.LG] UPDATED)
Although reinforcement learning (RL) has tremendous success in many fields, applying RL to real-world settings such as healthcare is challenging when the reward is hard to specify and no exploration is allowed. In this work, we focus on recovering clinicians' rewards in treating patients. We incorporate the what-if reasoning to explain the clinician's actions based on future outcomes. We use generalized additive models (GAMs) - a class of accurate, interpretable models - to recover the reward. In both simulation and a real-world hospital dataset, we show our model outperforms baselines. Finally, our model's explanations match several clinical guidelines when treating patients while we found the previously-used linear model often contradicts them.Adaptive Informative Path Planning Using Deep Reinforcement Learning for UAV-based Active Sensing. (arXiv:2109.13570v2 [cs.RO] UPDATED)
Aerial robots are increasingly being utilized for environmental monitoring and exploration. However, a key challenge is efficiently planning paths to maximize the information value of acquired data as an initially unknown environment is explored. To address this, we propose a new approach for informative path planning based on deep reinforcement learning (RL). Combining recent advances in RL and robotic applications, our method combines tree search with an offline-learned neural network predicting informative sensing actions. We introduce several components making our approach applicable for robotic tasks with high-dimensional state and large action spaces. By deploying the trained network during a mission, our method enables sample-efficient online replanning on platforms with limited computational resources. Simulations show that our approach performs on par with existing methods while reducing runtime by 8-10x. We validate its performance using real-world surface temperature data.Label-Aware Ranked Loss for robust People Counting using Automotive in-cabin Radar. (arXiv:2110.05876v2 [eess.SP] UPDATED)
In this paper, we introduce the Label-Aware Ranked loss, a novel metric loss function. Compared to the state-of-the-art Deep Metric Learning losses, this function takes advantage of the ranked ordering of the labels in regression problems. To this end, we first show that the loss minimises when datapoints of different labels are ranked and laid at uniform angles between each other in the embedding space. Then, to measure its performance, we apply the proposed loss on a regression task of people counting with a short-range radar in a challenging scenario, namely a vehicle cabin. The introduced approach improves the accuracy as well as the neighboring labels accuracy up to 83.0% and 99.9%: An increase of 6.7%and 2.1% on state-of-the-art methods, respectively.Scenario generation for market risk models using generative neural networks. (arXiv:2109.10072v2 [cs.LG] UPDATED)
In this research, we show how to expand existing approaches of using generative adversarial networks (GANs) as economic scenario generators (ESG) to a whole internal model - with enough risk factors to model the full band-width of investments for an insurance company and for a one year time horizon as required in Solvency 2. For validation of this approach as well as for optimization of GAN architecture, we provide a consistent, data-driven framework using existing evaluation measures based on nearest neighbor distances and a newly developed measure for the detection of the memorizing effect. Finally, we demonstrate that the results of a GAN-based ESG are similar to regulatory approved internal models in Europe. Therefore, GAN-based models can be seen as an assumption-free data-driven alternative way of market risk modeling.R2D2: Recursive Transformer based on Differentiable Tree for Interpretable Hierarchical Language Modeling. (arXiv:2107.00967v2 [cs.CL] UPDATED)
Human language understanding operates at multiple levels of granularity (e.g., words, phrases, and sentences) with increasing levels of abstraction that can be hierarchically combined. However, existing deep models with stacked layers do not explicitly model any sort of hierarchical process. This paper proposes a recursive Transformer model based on differentiable CKY style binary trees to emulate the composition process. We extend the bidirectional language model pre-training objective to this architecture, attempting to predict each word given its left and right abstraction nodes. To scale up our approach, we also introduce an efficient pruned tree induction algorithm to enable encoding in just a linear number of composition steps. Experimental results on language modeling and unsupervised parsing show the effectiveness of our approach.Extraction and Analysis of Highway On-Ramp Merging Scenarios from Naturalistic Trajectory Data. (arXiv:2104.05661v2 [cs.LG] UPDATED)
Connected and Automated Vehicles (CAVs) are envisioned to transform the future industrial and private transportation sectors. However, due to the system's enormous complexity, functional verification and validation of safety aspects are essential before the technology merges into the public domain. Therefore, in recent years, a scenario-driven approach has gained acceptance, emphasizing the requirement of a solid data basis of scenarios. The large-scale research facility Test Bed Lower Saxony (TFNDS) enables the provision of ample information for a database of scenarios on highways. For that purpose, however, the scenarios of interest must be identified and extracted from the collected Naturalistic Trajectory Data (NTD). This work addresses this problem and proposes a methodology for onramp scenario extraction, enabling scenario categorization and assessment. An Hidden Markov Model (HMM) and Dynamic Time Warping (DTW) is utilized for extraction and a decision tree with the Surrogate Measure of Safety (SMoS) Post Enroachment Time (PET) for categorization and assessment. The efficacy of the approach is shown with a dataset of NTD collected on the TFNDS.Reinforcement learning for pursuit and evasion of microswimmers at low Reynolds number. (arXiv:2106.08609v3 [physics.flu-dyn] UPDATED)
We consider a model of two competing microswimming agents engaged in a pursue-evasion task within a low-Reynolds-number environment. Agents can only perform simple maneuvers and sense hydrodynamic disturbances, which provide ambiguous (partial) information about the opponent's position and motion. We frame the problem as a zero-sum game: The pursuer has to capture the evader in the shortest time, while the evader aims at deferring capture as long as possible. We show that the agents, trained via adversarial reinforcement learning, are able to overcome partial observability by discovering increasingly complex sequences of moves and countermoves that outperform known heuristic strategies and exploit the hydrodynamic environment.What Makes Transfer Learning Work For Medical Images: Feature Reuse & Other Factors. (arXiv:2203.01825v1 [cs.LG])
Transfer learning is a standard technique to transfer knowledge from one domain to another. For applications in medical imaging, transfer from ImageNet has become the de-facto approach, despite differences in the tasks and image characteristics between the domains. However, it is unclear what factors determine whether - and to what extent - transfer learning to the medical domain is useful. The long-standing assumption that features from the source domain get reused has recently been called into question. Through a series of experiments on several medical image benchmark datasets, we explore the relationship between transfer learning, data size, the capacity and inductive bias of the model, as well as the distance between the source and target domain. Our findings suggest that transfer learning is beneficial in most cases, and we characterize the important role feature reuse plays in its success.Application of Ghost-DeblurGAN to Fiducial Marker Detection. (arXiv:2109.03379v3 [eess.IV] CROSS LISTED)
Feature extraction or localization based on the fiducial marker could fail due to motion blur in real-world robotic applications. To solve this problem, a lightweight generative adversarial network, named Ghost-DeblurGAN, for real-time motion deblurring is developed in this paper. Furthermore, on account that there is no existing deblurring benchmark for such task, a new large-scale dataset, YorkTag, is proposed that provides pairs of sharp/blurred images containing fiducial markers. With the proposed model trained and tested on YorkTag, it is demonstrated that when applied along with fiducial marker systems to motion-blurred images, Ghost-DeblurGAN improves the marker detection significantly. The datasets and codes used in this paper are available at: https://github.com/York-SDCNLab/Ghost-DeblurGAN.Rapid Convergence of the Unadjusted Langevin Algorithm: Isoperimetry Suffices. (arXiv:1903.08568v4 [cs.DS] UPDATED)
We study the Unadjusted Langevin Algorithm (ULA) for sampling from a probability distribution $\nu = e^{-f}$ on $\mathbb{R}^n$. We prove a convergence guarantee in Kullback-Leibler (KL) divergence assuming $\nu$ satisfies a log-Sobolev inequality and the Hessian of $f$ is bounded. Notably, we do not assume convexity or bounds on higher derivatives. We also prove convergence guarantees in R\'enyi divergence of order $q > 1$ assuming the limit of ULA satisfies either the log-Sobolev or Poincar\'e inequality. We also prove a bound on the bias of the limiting distribution of ULA assuming third-order smoothness of $f$, without requiring isoperimetry.A Brief Overview of Unsupervised Neural Speech Representation Learning. (arXiv:2203.01829v1 [eess.AS])
Unsupervised representation learning for speech processing has matured greatly in the last few years. Work in computer vision and natural language processing has paved the way, but speech data offers unique challenges. As a result, methods from other domains rarely translate directly. We review the development of unsupervised representation learning for speech over the last decade. We identify two primary model categories: self-supervised methods and probabilistic latent variable models. We describe the models and develop a comprehensive taxonomy. Finally, we discuss and compare models from the two categories.Improving Non-native Word-level Pronunciation Scoring with Phone-level Mixup Data Augmentation and Multi-source Information. (arXiv:2203.01826v1 [eess.AS])
Deep learning-based pronunciation scoring models highly rely on the availability of the annotated non-native data, which is costly and has scalability issues. To deal with the data scarcity problem, data augmentation is commonly used for model pretraining. In this paper, we propose a phone-level mixup, a simple yet effective data augmentation method, to improve the performance of word-level pronunciation scoring. Specifically, given a phoneme sequence from lexicon, the artificial augmented word sample can be generated by randomly sampling from the corresponding phone-level features in training data, while the word score is the average of their GOP scores. Benefit from the arbitrary phone-level combination, the mixup is able to generate any word with various pronunciation scores. Moreover, we utilize multi-source information (e.g., MFCC and deep features) to further improve the scoring system performance. The experiments conducted on the Speechocean762 show that the proposed system outperforms the baseline by adding the mixup data for pretraining, with Pearson correlation coefficients (PCC) increasing from 0.567 to 0.61. The results also indicate that proposed method achieves similar performance by using 1/10 unlabeled data of baseline. In addition, the experimental results also demonstrate the efficiency of our proposed multi-source approach.Generative Modeling for Low Dimensional Speech Attributes with Neural Spline Flows. (arXiv:2203.01786v1 [cs.SD])
Despite recent advances in generative modeling for text-to-speech synthesis, these models do not yet have the same fine-grained adjustability of pitch-conditioned deterministic models such as FastPitch and FastSpeech2. Pitch information is not only low-dimensional, but also discontinuous, making it particularly difficult to model in a generative setting. Our work explores several techniques for handling the aforementioned issues in the context of Normalizing Flow models. We also find this problem to be very well suited for Neural Spline flows, which is a highly expressive alternative to the more common affine-coupling mechanism in Normalizing Flows.Accelerated SGD for Non-Strongly-Convex Least Squares. (arXiv:2203.01744v1 [cs.LG])
We consider stochastic approximation for the least squares regression problem in the non-strongly convex setting. We present the first practical algorithm that achieves the optimal prediction error rates in terms of dependence on the noise of the problem, as $O(d/t)$ while accelerating the forgetting of the initial conditions to $O(d/t^2)$. Our new algorithm is based on a simple modification of the accelerated gradient descent. We provide convergence results for both the averaged and the last iterate of the algorithm. In order to describe the tightness of these new bounds, we present a matching lower bound in the noiseless setting and thus show the optimality of our algorithm.ADPCM with nonlinear prediction. (arXiv:2203.01818v1 [eess.AS])
Many speech coders are based on linear prediction coding (LPC), nevertheless with LPC is not possible to model the nonlinearities present in the speech signal. Because of this there is a growing interest for nonlinear techniques. In this paper we discuss ADPCM schemes with a nonlinear predictor based on neural nets, which yields an increase of 1-2.5dB in the SEGSNR over classical methods. This paper will discuss the block-adaptive and sample-adaptive predictions.Speech segmentation using multilevel hybrid filters. (arXiv:2203.01819v1 [eess.AS])
A novel approach for speech segmentation is proposed, based on Multilevel Hybrid (mean/min) Filters (MHF) with the following features: An accurate transition location. Good performance in noisy environments (gaussian and impulsive noise). The proposed method is based on spectral changes, with the goal of segmenting the voice into homogeneous acoustic segments. This algorithm is being used for phoneticallysegmented speech coder, with successful results.Detection of Word Adversarial Examples in Text Classification: Benchmark and Baseline via Robust Density Estimation. (arXiv:2203.01677v1 [cs.CL])
Word-level adversarial attacks have shown success in NLP models, drastically decreasing the performance of transformer-based models in recent years. As a countermeasure, adversarial defense has been explored, but relatively few efforts have been made to detect adversarial examples. However, detecting adversarial examples may be crucial for automated tasks (e.g. review sentiment analysis) that wish to amass information about a certain population and additionally be a step towards a robust defense system. To this end, we release a dataset for four popular attack methods on four datasets and four models to encourage further research in this field. Along with it, we propose a competitive baseline based on density estimation that has the highest AUC on 29 out of 30 dataset-attack-model combinations. Source code is available in https://github.com/anoymous92874838/text-adv-detection.On Practical Reinforcement Learning: Provable Robustness, Scalability, and Statistical Efficiency. (arXiv:2203.01758v1 [cs.LG])
This thesis rigorously studies fundamental reinforcement learning (RL) methods in modern practical considerations, including robust RL, distributional RL, and offline RL with neural function approximation. The thesis first prepares the readers with an overall overview of RL and key technical background in statistics and optimization. In each of the settings, the thesis motivates the problems to be studied, reviews the current literature, provides computationally efficient algorithms with provable efficiency guarantees, and concludes with future research directions. The thesis makes fundamental contributions to the three settings above, both algorithmically, theoretically, and empirically, while staying relevant to practical considerations.An Effective Graph Learning based Approach for Temporal Link Prediction: The First Place of WSDM Cup 2022. (arXiv:2203.01820v1 [cs.SI])
Temporal link prediction, as one of the most crucial work in temporal graphs, has attracted lots of attention from the research area. The WSDM Cup 2022 seeks for solutions that predict the existence probabilities of edges within time spans over temporal graph. This paper introduces the solution of AntGraph, which wins the 1st place in the competition. We first analysis the theoretical upper-bound of the performance by removing temporal information, which implies that only structure and attribute information on the graph could achieve great performance. Based on this hypothesis, then we introduce several well-designed features. Finally, experiments conducted on the competition datasets show the superiority of our proposal, which achieved AUC score of 0.666 on dataset A and 0.902 on dataset B, the ablation studies also prove the efficiency of each feature. Code is publicly available at https://github.com/im0qianqian/WSDM2022TGP-AntGraph.The Best of Both Worlds: Reinforcement Learning with Logarithmic Regret and Policy Switches. (arXiv:2203.01491v1 [cs.LG])
In this paper, we study the problem of regret minimization for episodic Reinforcement Learning (RL) both in the model-free and the model-based setting. We focus on learning with general function classes and general model classes, and we derive results that scale with the eluder dimension of these classes. In contrast to the existing body of work that mainly establishes instance-independent regret guarantees, we focus on the instance-dependent setting and show that the regret scales logarithmically with the horizon $T$, provided that there is a gap between the best and the second best action in every state. In addition, we show that such a logarithmic regret bound is realizable by algorithms with $O(\log T)$ switching cost (also known as adaptivity complexity). In other words, these algorithms rarely switch their policy during the course of their execution. Finally, we complement our results with lower bounds which show that even in the tabular setting, we cannot hope for regret guarantees lower than $o(\log T)$.Modularity of the ABCD Random Graph Model with Community Structure. (arXiv:2203.01480v1 [cs.SI])
The Artificial Benchmark for Community Detection (ABCD) graph is a random graph model with community structure and power-law distribution for both degrees and community sizes. The model generates graphs with similar properties as the well-known LFR one, and its main parameter $\xi$ can be tuned to mimic its counterpart in the LFR model, the mixing parameter $\mu$. In this paper, we investigate various theoretical asymptotic properties of the ABCD model. In particular, we analyze the modularity function, arguably, the most important graph property of networks in the context of community detection. Indeed, the modularity function is often used to measure the presence of community structure in networks. It is also used as a quality function in many community detection algorithms, including the widely used Louvain algorithm.Physics-Data Driven Machine Learning Based Model: A Hybrid Way for Nonlinear, Dynamic, and Open-loop Identification of IPMC Soft Artificial Muscles. (arXiv:2203.01616v1 [cs.LG])
Ionic Polymer Metal Composites (IPMCs) are one of the most preferred choices among biocompatible materials for industrial and biomedical applications. Despite their advantages, some of their drawbacks include non-linear and hysteretic behavior, which complicates the modeling process. In previous works, usually autoregressive models were used to predict the behavior of an IPMC actuator. The main drawback of using an autoregressive model is that it cannot be used in mobile and real-time applications. In this study, we proposed a hybrid analytical intelligent model for an IPMC actuator. The most outstanding feature of this model is its non-autoregressive structure. The hybrid concept proposed in this study can be generalized to various problems other than IPMCs. The structure used in this work comprises an analytical model and a deep neural network, providing a non-linear, dynamic, and non-autoregressive model for the IPMC actuator. Lastly, the average NMSE achieved using the proposed hybrid model is 9.5781e-04 showing a significant drop in the error rate compared to other non-autoregressive structures.AdaFamily: A family of Adam-like adaptive gradient methods. (arXiv:2203.01603v1 [cs.LG])
We propose AdaFamily, a novel method for training deep neural networks. It is a family of adaptive gradient methods and can be interpreted as sort of a blend of the optimization algorithms Adam, AdaBelief and AdaMomentum. We perform experiments on standard datasets for image classification, demonstrating that our proposed method outperforms these algorithms.Early Time-Series Classification Algorithms: An Empirical Comparison. (arXiv:2203.01628v1 [cs.LG])
Early Time-Series Classification (ETSC) is the task of predicting the class of incoming time-series by observing as few measurements as possible. Such methods can be employed to obtain classification forecasts in many time-critical applications. However, available techniques are not equally suitable for every problem, since differentiations in the data characteristics can impact algorithm performance in terms of earliness, accuracy, F1-score, and training time. We evaluate six existing ETSC algorithms on publicly available data, as well as on two newly introduced datasets originating from the life sciences and maritime domains. Our goal is to provide a framework for the evaluation and comparison of ETSC algorithms and to obtain intuition on how such approaches perform on real-life applications. The presented framework may also serve as a benchmark for new related techniques.Fully-Connected Network on Noncompact Symmetric Space and Ridgelet Transform based on Helgason-Fourier Analysis. (arXiv:2203.01631v1 [cs.LG])
Neural network on Riemannian symmetric space such as hyperbolic space and the manifold of symmetric positive definite (SPD) matrices is an emerging subject of research in geometric deep learning. Based on the well-established framework of the Helgason-Fourier transform on the noncompact symmetric space, we present a fully-connected network and its associated ridgelet transform on the noncompact symmetric space, covering the hyperbolic neural network (HNN) and the SPDNet as special cases. The ridgelet transform is an analysis operator of a depth-2 continuous network spanned by neurons, namely, it maps an arbitrary given function to the weights of a network. Thanks to the coordinate-free reformulation, the role of nonlinear activation functions is revealed to be a wavelet function, and the reconstruction formula directly yields the universality of the proposed networks.Improving X-ray Diagnostics through Eye-Tracking and XR. (arXiv:2203.01643v1 [cs.HC])
There is a growing need to assist radiologists in performing X-ray readings and diagnoses fast, comfortably, and effectively. As radiologists strive to maximize productivity, it is essential to consider the impact of reading rooms in interpreting complex examinations and ensure that higher volume and reporting speeds do not compromise patient outcomes. Virtual Reality (VR) is a disruptive technology for clinical practice in assessing X-ray images. We argue that conjugating eye-tracking with VR devices and Machine Learning may overcome obstacles posed by inadequate ergonomic postures and poor room conditions that often cause erroneous diagnostics when professionals examine digital images.A shallow physics-informed neural network for solving partial differential equations on surfaces. (arXiv:2203.01581v1 [math.NA])
In this paper, we introduce a mesh-free physics-informed neural network for solving partial differential equations on surfaces. Based on the idea of embedding techniques, we write the underlying surface differential equations using conventional Cartesian differential operators. With the aid of level set function, the surface geometrical quantities, such as the normal and mean curvature of the surface, can be computed directly and used in our surface differential expressions. So instead of imposing the normal extension constraints used in literature, we take the whole Cartesian differential expressions into account in our loss function. Meanwhile, we adopt a completely shallow (one hidden layer) network so the present model is easy to implement and train. We perform a series of numerical experiments on both stationary and time-dependent partial differential equations on complicated surface geometries. The result shows that, with just a few hundred trainable parameters, our network model is able to achieve high predictive accuracy.Reconstruction of univariate functions from directional persistence diagrams. (arXiv:2203.01894v1 [math.AT])
We describe a method for approximating a single-variable function $f$ using persistence diagrams of sublevel sets of $f$ from height functions in different directions. We provide algorithms for the piecewise linear case and for the smooth case. Three directions suffice to locate all local maxima and minima of a piecewise linear continuous function from its collection of directional persistence diagrams, while five directions are needed in the case of smooth functions with non-degenerate critical points. Our approximation of functions by means of persistence diagrams is motivated by a study of importance attribution in machine learning, where one seeks to reduce the number of critical points of signal functions without a significant loss of information for a neural network classifier.Socially Aware Robot Crowd Navigation with Interaction Graphs and Human Trajectory Prediction. (arXiv:2203.01821v1 [cs.RO])
We study the problem of safe and socially aware robot navigation in dense and interactive human crowds. Previous works use simplified methods to model the personal spaces of pedestrians and ignore the social compliance of the robot behaviors. In this paper, we provide a more accurate representation of personal zones of walking pedestrians with their future trajectories. The predicted personal zones are incorporated into a reinforcement learning framework to prevent the robot from intruding into the personal zones. To learn socially aware navigation policies, we propose a novel recurrent graph neural network with attention mechanisms to capture the interactions among agents through space and time. We demonstrate that our method enables the robot to achieve good navigation performance and non-invasiveness in challenging crowd navigation scenarios. We successfully transfer the policy learned in the simulator to a real-world TurtleBot 2i.BERT Learns to Teach: Knowledge Distillation with Meta Learning. (arXiv:2106.04570v2 [cs.LG] UPDATED)
We present Knowledge Distillation with Meta Learning (MetaDistil), a simple yet effective alternative to traditional knowledge distillation (KD) methods where the teacher model is fixed during training. We show the teacher network can learn to better transfer knowledge to the student network (i.e., learning to teach) with the feedback from the performance of the distilled student network in a meta learning framework. Moreover, we introduce a pilot update mechanism to improve the alignment between the inner-learner and meta-learner in meta learning algorithms that focus on an improved inner-learner. Experiments on various benchmarks show that MetaDistil can yield significant improvements compared with traditional KD algorithms and is less sensitive to the choice of different student capacity and hyperparameters, facilitating the use of KD on different tasks and models.Colon Nuclei Instance Segmentation using a Probabilistic Two-Stage Detector. (arXiv:2203.01321v1 [eess.IV])
Cancer is one of the leading causes of death in the developed world. Cancer diagnosis is performed through the microscopic analysis of a sample of suspicious tissue. This process is time consuming and error prone, but Deep Learning models could be helpful for pathologists during cancer diagnosis. We propose to change the CenterNet2 object detection model to also perform instance segmentation, which we call SegCenterNet2. We train SegCenterNet2 in the CoNIC challenge dataset and show that it performs better than Mask R-CNN in the competition metrics. ( 2 min )Stable and Semi-stable Sampling Approaches for Continuously Used Samples. (arXiv:2203.01381v1 [cs.IR])
Information retrieval systems are usually measured by labeling the relevance of results corresponding to a sample of user queries. In practical search engines, such measurement needs to be performed continuously, such as daily or weekly. This creates a trade-off between (a) representativeness of query sample to current query traffic of the product; (b) labeling cost: if we keep the same query sample, results would be similar allowing us to reuse their labels; and (c) overfitting caused by continuous usage of same query sample. In this paper we explicitly formulate this tradeoff, propose two new variants -- Stable and Semi-stable -- to simple and weighted random sampling and show that they outperform existing approaches for the continuous usage settings, including monitoring/debugging search engine or comparing ranker candidates. ( 2 min )Scalable Bayesian Optimization Using Vecchia Approximations of Gaussian Processes. (arXiv:2203.01459v1 [cs.LG])
Bayesian optimization is a technique for optimizing black-box target functions. At the core of Bayesian optimization is a surrogate model that predicts the output of the target function at previously unseen inputs to facilitate the selection of promising input values. Gaussian processes (GPs) are commonly used as surrogate models but are known to scale poorly with the number of observations. We adapt the Vecchia approximation, a popular GP approximation from spatial statistics, to enable scalable high-dimensional Bayesian optimization. We develop several improvements and extensions, including training warped GPs using mini-batch gradient descent, approximate neighbor search, and selecting multiple input values in parallel. We focus on the use of our warped Vecchia GP in trust-region Bayesian optimization via Thompson sampling. On several test functions and on two reinforcement-learning problems, our methods compared favorably to the state of the art. ( 2 min )Detecting Chronic Kidney Disease(CKD) at the Initial Stage: A Novel Hybrid Feature-selection Method and Robust Data Preparation Pipeline for Different ML Techniques. (arXiv:2203.01394v1 [cs.LG])
Chronic Kidney Disease (CKD) has infected almost 800 million people around the world. Around 1.7 million people die each year because of it. Detecting CKD in the initial stage is essential for saving millions of lives. Many researchers have applied distinct Machine Learning (ML) methods to detect CKD at an early stage, but detailed studies are still missing. We present a structured and thorough method for dealing with the complexities of medical data with optimal performance. Besides, this study will assist researchers in producing clear ideas on the medical data preparation pipeline. In this paper, we applied KNN Imputation to impute missing values, Local Outlier Factor to remove outliers, SMOTE to handle data imbalance, K-stratified K-fold Cross-validation to validate the ML models, and a novel hybrid feature selection method to remove redundant features. Applied algorithms in this study are Support Vector Machine, Gaussian Naive Bayes, Decision Tree, Random Forest, Logistic Regression, K-Nearest Neighbor, Gradient Boosting, Adaptive Boosting, and Extreme Gradient Boosting. Finally, the Random Forest can detect CKD with 100% accuracy without any data leakage. ( 2 min )Successful Recovery of an Observed Meteorite Fall Using Drones and Machine Learning. (arXiv:2203.01466v1 [astro-ph.EP])
We report the first-time recovery of a fresh meteorite fall using a drone and a machine learning algorithm. A fireball on the 1st April 2021 was observed over Western Australia by the Desert Fireball Network, for which a fall area was calculated for the predicted surviving mass. A search team arrived on site and surveyed 5.1 km2 area over a 4-day period. A convolutional neural network, trained on previously-recovered meteorites with fusion crusts, processed the images on our field computer after each flight. meteorite candidates identified by the algorithm were sorted by team members using two user interfaces to eliminate false positives. Surviving candidates were revisited with a smaller drone, and imaged in higher resolution, before being eliminated or finally being visited in-person. The 70 g meteorite was recovered within 50 m of the calculated fall line using, demonstrating the effectiveness of this methodology which will facilitate the efficient collection of many more observed meteorite falls. ( 2 min )Precise Stock Price Prediction for Optimized Portfolio Design Using an LSTM Model. (arXiv:2203.01326v1 [q-fin.PM])
Accurate prediction of future prices of stocks is a difficult task to perform. Even more challenging is to design an optimized portfolio of stocks with the identification of proper weights of allocation to achieve the optimized values of return and risk. We present optimized portfolios based on the seven sectors of the Indian economy. The past prices of the stocks are extracted from the web from January 1, 2016, to December 31, 2020. Optimum portfolios are designed on the selected seven sectors. An LSTM regression model is also designed for predicting future stock prices. Five months after the construction of the portfolios, i.e., on June 1, 2021, the actual and predicted returns and risks of each portfolio are computed. The predicted and the actual returns indicate the very high accuracy of the LSTM model. ( 2 min )Rethinking the role of normalization and residual blocks for spiking neural networks. (arXiv:2203.01544v1 [cs.NE])
Biologically inspired spiking neural networks (SNNs) are widely used to realize ultralow-power energy consumption. However, deep SNNs are not easy to train due to the excessive firing of spiking neurons in the hidden layers. To tackle this problem, we propose a novel but simple normalization technique called postsynaptic potential normalization. This normalization removes the subtraction term from the standard normalization and uses the second raw moment instead of the variance as the division term. The spike firing can be controlled, enabling the training to proceed appropriating, by conducting this simple normalization to the postsynaptic potential. The experimental results show that SNNs with our normalization outperformed other models using other normalizations. Furthermore, through the pre-activation residual blocks, the proposed model can train with more than 100 layers without other special techniques dedicated to SNNs. ( 2 min )Near-Optimal Correlation Clustering with Privacy. (arXiv:2203.01440v1 [cs.LG])
Correlation clustering is a central problem in unsupervised learning, with applications spanning community detection, duplicate detection, automated labelling and many more. In the correlation clustering problem one receives as input a set of nodes and for each node a list of co-clustering preferences, and the goal is to output a clustering that minimizes the disagreement with the specified nodes' preferences. In this paper, we introduce a simple and computationally efficient algorithm for the correlation clustering problem with provable privacy guarantees. Our approximation guarantees are stronger than those shown in prior work and are optimal up to logarithmic factors. ( 2 min )LILE: Look In-Depth before Looking Elsewhere -- A Dual Attention Network using Transformers for Cross-Modal Information Retrieval in Histopathology Archives. (arXiv:2203.01445v1 [cs.CV])
The volume of available data has grown dramatically in recent years in many applications. Furthermore, the age of networks that used multiple modalities separately has practically ended. Therefore, enabling bidirectional cross-modality data retrieval capable of processing has become a requirement for many domains and disciplines of research. This is especially true in the medical field, as data comes in a multitude of types, including various types of images and reports as well as molecular data. Most contemporary works apply cross attention to highlight the essential elements of an image or text in relation to the other modalities and try to match them together. However, regardless of their importance in their own modality, these approaches usually consider features of each modality equally. In this study, self-attention as an additional loss term will be proposed to enrich the internal representation provided into the cross attention module. This work suggests a novel architecture with a new loss term to help represent images and texts in the joint latent space. Experiment results on two benchmark datasets, i.e. MS-COCO and ARCH, show the effectiveness of the proposed method. ( 2 min ) -
Open
Early Time-Series Classification Algorithms: An Empirical Comparison. (arXiv:2203.01628v1 [cs.LG])
Early Time-Series Classification (ETSC) is the task of predicting the class of incoming time-series by observing as few measurements as possible. Such methods can be employed to obtain classification forecasts in many time-critical applications. However, available techniques are not equally suitable for every problem, since differentiations in the data characteristics can impact algorithm performance in terms of earliness, accuracy, F1-score, and training time. We evaluate six existing ETSC algorithms on publicly available data, as well as on two newly introduced datasets originating from the life sciences and maritime domains. Our goal is to provide a framework for the evaluation and comparison of ETSC algorithms and to obtain intuition on how such approaches perform on real-life applications. The presented framework may also serve as a benchmark for new related techniques.Accelerated SGD for Non-Strongly-Convex Least Squares. (arXiv:2203.01744v1 [cs.LG])
We consider stochastic approximation for the least squares regression problem in the non-strongly convex setting. We present the first practical algorithm that achieves the optimal prediction error rates in terms of dependence on the noise of the problem, as $O(d/t)$ while accelerating the forgetting of the initial conditions to $O(d/t^2)$. Our new algorithm is based on a simple modification of the accelerated gradient descent. We provide convergence results for both the averaged and the last iterate of the algorithm. In order to describe the tightness of these new bounds, we present a matching lower bound in the noiseless setting and thus show the optimality of our algorithm.Scalable Bayesian Optimization Using Vecchia Approximations of Gaussian Processes. (arXiv:2203.01459v1 [cs.LG])
Bayesian optimization is a technique for optimizing black-box target functions. At the core of Bayesian optimization is a surrogate model that predicts the output of the target function at previously unseen inputs to facilitate the selection of promising input values. Gaussian processes (GPs) are commonly used as surrogate models but are known to scale poorly with the number of observations. We adapt the Vecchia approximation, a popular GP approximation from spatial statistics, to enable scalable high-dimensional Bayesian optimization. We develop several improvements and extensions, including training warped GPs using mini-batch gradient descent, approximate neighbor search, and selecting multiple input values in parallel. We focus on the use of our warped Vecchia GP in trust-region Bayesian optimization via Thompson sampling. On several test functions and on two reinforcement-learning problems, our methods compared favorably to the state of the art.On Practical Reinforcement Learning: Provable Robustness, Scalability, and Statistical Efficiency. (arXiv:2203.01758v1 [cs.LG])
This thesis rigorously studies fundamental reinforcement learning (RL) methods in modern practical considerations, including robust RL, distributional RL, and offline RL with neural function approximation. The thesis first prepares the readers with an overall overview of RL and key technical background in statistics and optimization. In each of the settings, the thesis motivates the problems to be studied, reviews the current literature, provides computationally efficient algorithms with provable efficiency guarantees, and concludes with future research directions. The thesis makes fundamental contributions to the three settings above, both algorithmically, theoretically, and empirically, while staying relevant to practical considerations.Rapid Convergence of the Unadjusted Langevin Algorithm: Isoperimetry Suffices. (arXiv:1903.08568v4 [cs.DS] UPDATED)
We study the Unadjusted Langevin Algorithm (ULA) for sampling from a probability distribution $\nu = e^{-f}$ on $\mathbb{R}^n$. We prove a convergence guarantee in Kullback-Leibler (KL) divergence assuming $\nu$ satisfies a log-Sobolev inequality and the Hessian of $f$ is bounded. Notably, we do not assume convexity or bounds on higher derivatives. We also prove convergence guarantees in R\'enyi divergence of order $q > 1$ assuming the limit of ULA satisfies either the log-Sobolev or Poincar\'e inequality. We also prove a bound on the bias of the limiting distribution of ULA assuming third-order smoothness of $f$, without requiring isoperimetry. ( 2 min )
